M = Syntax + Prosody: A syntactic-prosodic labelling scheme for large spontaneous speech databases

In automatic speech understanding, division of continuous running speech into syntactic chunks is a great problem. Syntactic boundaries are often marked by prosodic means. For the training of statistical models for prosodic boundaries large databases are necessary. For the German Verbmobil (VM) project (automatic speech-to-speech translation), we developed a syntactic±prosodic labelling scheme where di€erent types of syntactic boundaries are labelled for a large spontaneous speech corpus. This labelling scheme is presented and compared with other labelling schemes for perceptual±prosodic, syntactic, and dialogue act boundaries. Interlabeller consistencies and estimation of e€ort needed are discussed. We compare the results of classi®ers (multi-layer perceptrons (MLPs) and n-gram language models) trained on these syntactic±prosodic boundary labels with classi®ers trained on perceptual±prosodic and pure syntactic labels. The main advantage of the rough syntactic±prosodic labels presented in this paper is that large amounts of data can be labelled with relatively little e€ort. The classi®ers trained with these labels turned out to be superior with respect to purely prosodic or syntactic labelling schemes, yielding recognition rates of up to 96% for the two-class-problem `boundary versus no boundary'. The use of boundary information leads to a marked improvement in the syntactic processing of the VM system. Ó 1998 Elsevier Science B.V. All rights reserved.